{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GEE score tests\n",
    "\n",
    "This notebook uses simulation to demonstrate robust GEE score tests.  These tests can be used in a GEE analysis to compare nested hypotheses about the mean structure.  The tests are robust to miss-specification of the working correlation model, and to certain forms of misspecification of the variance structure (e.g. as captured by the scale parameter in a quasi-Poisson analysis).\n",
    "\n",
    "The data are simulated as clusters, where there is dependence within but not between clusters.  The cluster-wise dependence is induced using a copula approach.  The data marginally follow a negative binomial (gamma/Poisson) mixture.\n",
    "\n",
    "The level and power of the tests are considered below to assess the performance of the tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from scipy.stats.distributions import norm, poisson\n",
    "import statsmodels.api as sm\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function defined in the following cell uses a copula approach to simulate correlated random values that marginally follow a negative binomial distribution.  The input parameter `u` is an array of values in (0, 1).  The elements of `u` must be marginally uniformly distributed on (0, 1).  Correlation in `u` will induce correlations in the returned negative binomial values.  The array parameter `mu` gives the marginal means, and the scalar parameter `scale` defines the mean/variance relationship (the variance is `scale` times the mean).  The lengths of `u` and `mu` must be the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 1
   },
   "outputs": [],
   "source": [
    "def negbinom(u, mu, scale):\n",
    "    p = (scale - 1) / scale\n",
    "    r = mu * (1 - p) / p\n",
    "    x = np.random.gamma(r, p / (1 - p), len(u))\n",
    "    return poisson.ppf(u, mu=x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below are some parameters that govern the data used in the simulation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sample size\n",
    "n = 1000\n",
    "\n",
    "# Number of covariates (including intercept) in the alternative hypothesis model\n",
    "p = 5\n",
    "\n",
    "# Cluster size\n",
    "m = 10\n",
    "\n",
    "# Intraclass correlation (controls strength of clustering)\n",
    "r = 0.5\n",
    "\n",
    "# Group indicators\n",
    "grp = np.kron(np.arange(n/m), np.ones(m))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simulation uses a fixed design matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build a design matrix for the alternative (more complex) model\n",
    "x = np.random.normal(size=(n, p))\n",
    "x[:, 0] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The null design matrix is nested in the alternative design matrix.  It has rank two less than the alternative design matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x0 = x[:, 0:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The GEE score test is robust to dependence and overdispersion.  Here we set the overdispersion parameter.  The variance of the negative binomial distribution for each observation is equal to `scale` times its mean value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scale parameter for negative binomial distribution\n",
    "scale = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next cell, we set up the mean structures for the null and alternative models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The coefficients used to define the linear predictors\n",
    "coeff = [[4, 0.4, -0.2], [4, 0.4, -0.2, 0, -0.04]]\n",
    "\n",
    "# The linear predictors\n",
    "lp = [np.dot(x0, coeff[0]), np.dot(x, coeff[1])]\n",
    "\n",
    "# The mean values\n",
    "mu = [np.exp(lp[0]), np.exp(lp[1])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a function that carries out the simulation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# hyp = 0 is the null hypothesis, hyp = 1 is the alternative hypothesis.\n",
    "# cov_struct is a statsmodels covariance structure\n",
    "def dosim(hyp, cov_struct=None, mcrep=500):\n",
    "    \n",
    "    # Storage for the simulation results\n",
    "    scales = [[], []]\n",
    "    \n",
    "    # P-values from the score test\n",
    "    pv = []\n",
    "    \n",
    "    # Monte Carlo loop\n",
    "    for k in range(mcrep):\n",
    "\n",
    "        # Generate random \"probability points\" u  that are uniformly \n",
    "        # distributed, and correlated within clusters\n",
    "        z = np.random.normal(size=n)\n",
    "        u = np.random.normal(size=n//m)\n",
    "        u = np.kron(u, np.ones(m))\n",
    "        z = r*z +np.sqrt(1-r**2)*u\n",
    "        u = norm.cdf(z)\n",
    "\n",
    "        # Generate the observed responses\n",
    "        y = negbinom(u, mu=mu[hyp], scale=scale)\n",
    "\n",
    "        # Fit the null model\n",
    "        m0 = sm.GEE(y, x0, groups=grp, cov_struct=cov_struct, family=sm.families.Poisson())\n",
    "        r0 = m0.fit(scale='X2')\n",
    "        scales[0].append(r0.scale)\n",
    "        \n",
    "        # Fit the alternative model\n",
    "        m1 = sm.GEE(y, x, groups=grp, cov_struct=cov_struct, family=sm.families.Poisson())\n",
    "        r1 = m1.fit(scale='X2')\n",
    "        scales[1].append(r1.scale)\n",
    "        \n",
    "        # Carry out the score test\n",
    "        st = m1.compare_score_test(r0)\n",
    "        pv.append(st[\"p-value\"])\n",
    "\n",
    "    pv = np.asarray(pv)\n",
    "    rslt = [np.mean(pv), np.mean(pv < 0.1)]\n",
    "    \n",
    "    return rslt, scales"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the simulation using the independence working covariance structure.  We expect the mean to be around 0 under the null hypothesis, and much lower under the alternative hypothesis.  Similarly, we expect that under the null hypothesis, around 10% of the p-values are less than 0.1, and a much greater fraction of the p-values are less than 0.1 under the alternative hypothesis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rslt, scales = [], []\n",
    "\n",
    "for hyp in 0, 1:\n",
    "    s, t = dosim(hyp, sm.cov_struct.Independence())\n",
    "    rslt.append(s)\n",
    "    scales.append(t)\n",
    "    \n",
    "rslt = pd.DataFrame(rslt, index=[\"H0\", \"H1\"], columns=[\"Mean\", \"Prop(p<0.1)\"])\n",
    "\n",
    "print(rslt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we check to make sure that the scale parameter estimates are reasonable. We are assessing the robustness of the GEE score test to dependence and overdispersion, so here we are confirming that the overdispersion is present as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = plt.boxplot([scales[0][0], scales[0][1], scales[1][0], scales[1][1]])\n",
    "plt.ylabel(\"Estimated scale\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we conduct the same analysis using an exchangeable working correlation model.  Note that this will be slower than the example above using independent working correlation, so we use fewer Monte Carlo repetitions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rslt, scales = [], []\n",
    "\n",
    "for hyp in 0, 1:\n",
    "    s, t = dosim(hyp, sm.cov_struct.Exchangeable(), mcrep=100)\n",
    "    rslt.append(s)\n",
    "    scales.append(t)\n",
    "    \n",
    "rslt = pd.DataFrame(rslt, index=[\"H0\", \"H1\"], columns=[\"Mean\", \"Prop(p<0.1)\"])\n",
    "\n",
    "print(rslt)"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
